Method and apparatus for processing a point cloud

ABSTRACT

A method for encoding and decoding, an encoder and decoder for a point cloud. The method for encoding a point cloud to generate a bitstream of compressed point cloud data, in which the point cloud&#39;s geometry is represented by an octree-based structure with a plurality of nodes having parent-child relationships by recursively splitting a volumetric space containing the point cloud into sub-volumes each associated with a node of the octree-based structure, includes: determining an occupancy pattern for a parent node based on the occupancy of its child nodes; determining a planar context information for at least one of the child nodes; and entropy encoding/decoding the occupancy pattern parent node based on the determined planar context information to produce encoded/decoded data for the bitstream.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a U.S. national phase of International ApplicationNo. PCT/CN2020/084804, filed with the State Intellectual Property Officeof P. R. China on Apr. 14, 2020.

FIELD

The present application generally relates to point cloud compression.More specifically, the present application relates to a method ofencoding and decoding as well as an encoder and decoder for improvedentropy coding of point clouds.

BACKGROUND

As an alternative to 3D meshes, 3D point clouds have recently emerged asa popular representation of 3D media information. Use cases associatedwith point cloud data are very diverse and include:

-   -   3D assets in movie production,    -   3D assets for real-time 3D immersive telepresence or VR        applications,    -   3D free viewpoint video (for instance for sports viewing),    -   Geographical Information Systems (cartography),    -   Culture heritage (storage of fragile assets in digital form),    -   Autonomous driving (large scale 3D mapping of environment) . . .

A point cloud is a set of points in a 3D space, each with associatedattributes, e.g. color, material properties, etc. Point clouds can beused to reconstruct an object or a scene as a composition of suchpoints. They can be captured using multiple cameras and depth sensors invarious setups and may be made up of thousands up to billions of pointsin order to realistically represent reconstructed scenes.

For each points of a point cloud, its position (usually an X, Y, Zinformation coded as a floating point with 32 or 64 bits) and itsattributes (usually at least an RGB color coded in 24 bits) need to bestored. With sometimes billions of points in a point cloud, one caneasily understand that the raw data of a point cloud can be severalGigabytes of data: hence, there is a strong need for compressiontechnologies so as to reduce the amount of data required to represent apoint cloud.

Two different approaches were developed for point cloud compression:

First, in the Video based Point Cloud Compression (VPCC) approach, apoint cloud is compressed by performing multiple projections of it onthe 3 different axis X, Y, Z and on different depths so that all pointsare present in one projected image. Then the projected images areprocessed into patches (to eliminate redundancy) and re-arranged into afinal picture where additional metadata is used to translate pixelspositions into point positions in space. The compression is thenperformed using traditional image/video MPEG encoders. The advantage ofthis approach is that it reuses existing coders and it naturallysupports dynamic point clouds (using video coders) but this is hardlyusable for scarce point clouds and it is expected that the compressiongain would be higher with point clouds dedicated methods.

Second, in the Geometry based Point Cloud Compression (GPCC) approach,points positions (usually referred to as the geometry) and pointsattributes (color, transparency . . . ) are coded separately. In orderto code the geometry, an octree structure is used. The whole point cloudis fitted into a cube which is continuously split into eight sub-cubesuntil each of the sub-cubes contains only a single point. The positionof the points is therefore replaced by a tree of occupancy informationat every node. Since each cube has only 8 sub-cubes, 3 bits are enoughto code the occupancy and therefore for a tree of depth D, 3^(D) bitsare needed to code the position of a point. While this transformationalone is not enough to provide significant compression gain, it shouldbe noted that since it is a tree, many points share the same node valuesand thanks to the use of entropy coders, the amount of information canbe significantly reduced.

Thus, it is an object of the present invention to provide an encodingand decoding method as well as an encoder and decoder enabling improvedquick and accurate data compression of point clouds.

SUMMARY

In an aspect of the present invention, a method for encoding a pointcloud is provided to generate a bitstream of compressed point clouddata, in which the point cloud's geometry is represented by anoctree-based structure with a plurality of nodes having parent-childrelationships by recursively splitting a volumetric space containing thepoint cloud into sub-volumes each associated with a node of theoctree-based structure; the method includes the steps of: determining anoccupancy pattern for a parent node based on the occupancy of its childnodes;

determining a planar context information for at least one of the childnodes, in which the planar context information of a present child nodeis determined according to the occupancy pattern of the present parentnode and the occupancy pattern of at least one neighboring parent nodepositioned directly next to the present parent node; and

-   -   entropy encoding the occupancy pattern of the parent node based        on the determined planar context information to produce encoded        data for the bitstream.

In an aspect of the present invention a method for decoding a bitstreamof compressed point cloud data is provided to generate a reconstructedpoint cloud, in which the point cloud's geometry is represented by anoctree-based structure with a plurality of nodes having parent-childrelationships by recursively splitting a volumetric space containing thepoint cloud into sub-volumes each associated with a node of theoctree-bases structure, the method includes the steps of: entropydecoding the bitstream based on the planar context information of thepresent child node, in which the planar context information of thepresent child node is determined according to the occupancy pattern ofthe present parent node and the occupancy pattern of at least oneneighboring parent node positioned directly next to the present parentnode.

In an aspect of the present invention, an encoder is provided forencoding a point cloud to generate a bitstream of compressed point clouddata, in which the point cloud's geometry is represented by anoctree-based structure with a plurality of nodes having parent-childrelationships by recursively splitting a volumetric space containing thepoint cloud into sub-volumes each associated with a node of theoctree-based structure, the encoder including:

a processor and

a memory storage device, in which in the memory storage deviceinstructions executable by the processor are stored that, when executed,cause the processor to perform the method according to theabove-described methods for encoding and decoding.

In an aspect of the present invention, a decoder is provided fordecoding a bitstream of compressed point cloud data to generate areconstructed point cloud, in which the point cloud's geometry isrepresented by an octree-based structure with a plurality of nodeshaving parent-child relationships by recursively splitting a volumetricspace containing the point cloud into sub-volumes each associated with anode of the octree-bases structure, the decoder including:

a processor and

a memory storage device, in which in the memory storage deviceinstructions executable by the processor are stored that, when executed,cause the processor to perform the above-described method of decoding.

In an aspect of the present invention a non-transitory computer-readablestorage medium is provided storing processor-executed instructions that,when executed by a processor, cause the processor to perform theabove-described method of encoding and/or decoding.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made, by way of example, to the accompanyingdrawings which show example embodiments of the present application, andin which the Figures show:

FIG. 1 a block diagram showing a general view of the point cloudencoder,

FIG. 2 a block diagram showing a general view of the point clouddecoder,

FIG. 3 a schematic illustration of an octree data structure,

FIG. 4 numbering of the eight sub-nodes in each node,

FIG. 5 neighbors of a node with known occupancy information,

FIG. 6 neighboring nodes used for the three plane directions,

FIG. 7 flow charts illustrating the steps of a first and a secondembodiment of the present invention,

FIG. 8 a comparison of the current method of encoding and the presentinvention,

FIG. 9 a schematic illustration of an encoder device, and

FIG. 10 a schematic illustration of a decoder device.

DETAILED DESCRIPTION OF THE DRAWINGS

The present application describes methods of encoding and decoding pointclouds, and encoders and decoders for encoding and decoding pointclouds. A present parent node associated with a sub-volume is split intofurther sub-volumes, each further sub-volume corresponding to a childnode of the present parent node, and, at the encoder, an occupancypattern is determined for the present parent node based on occupancystatus of the child nodes. A planar context information is determinedfrom the occupancy pattern of the present parent node and an occupancypattern of a neighboring parent node positioned directly next to thepresent parent node. The entropy encoder encodes the occupancy patternbased on the determined planar context information to produce encodeddata for the bitstream. The decoder determines the same planar contextinformation and entropy decodes the bitstream to reconstruct theoccupancy pattern.

In an aspect of the present invention, a method for encoding a pointcloud is provided to generate a bitstream of compressed point clouddata, in which the point cloud's geometry is represented by anoctree-based structure with a plurality of nodes having parent-childrelationships by recursively splitting a volumetric space containing thepoint cloud into sub-volumes each associated with a node of theoctree-based structure; the method includes the steps of: determining anoccupancy pattern for a parent node based on the occupancy of its childnodes;

determining a planar context information for at least one of the childnodes, in which the planar context information of a present child nodeis determined according to the occupancy pattern of the present parentnode and the occupancy pattern of at least one neighboring parent nodepositioned directly next to the present parent node; and

entropy encoding the occupancy pattern of the parent node based on thedetermined planar context information to produce encoded data for thebitstream.

Therein, the present parent node and the neighboring parent node are onthe same depth D of the tree. Due to their position, the present parentnode and the neighboring parent node share a common surface. For thecurrent child node, being a direct child of the present parent node atdepth D+1 of the tree, the planar context information is determined independence on the occupancy pattern of the present parent node and theat least one neighboring parent node. Therein, the planar contextinformation includes information about a possible plane or surfacerepresented by the point cloud going through the current child node.Thus, information about the occupancy of at least one direct neighbor ofthe present parent node is considered as context information for theentropy encoding. As a consequence, more reliable predictions aboutoccupancy of the current child node can be made which can be used forincreased data reduction by the entropy encoder. In this manner, thecomplete tree is traversed to determine an occupancy pattern for eachnode and provides sufficient context information for the entropyencoder.

Preferably, the neighboring parent node is the node directly next to thepresent child node, sharing one surface with the present child node.

Preferably, the neighboring parent node is arranged along one of thegeometrical axes. If planar context information is evaluated along oneof the geometrical axes—X, Y, Z—then the neighboring parent node isarranged along this specific axis directly before or directly after thepresent parent node.

Preferably, the planar context information is determined along all threegeometrical axes. Thus, planes or surfaces represented by the points ofthe point cloud are evaluated along each axis.

Preferably, the present parent node is arranged relatively to theneighboring parent node in order of decoding. During decoding, the nodesof the octree-based structure are successively evaluated. Thus,information about occupancy patterns in the order of decoding before thepresent parent node is already accessible and thus can be used fordetermining the planar context information for decoding. In particular,the order of decoding is exactly inverse to the order of encoding.However, during encoding, the exact position of each point in the pointcloud is already known.

Preferably, the occupancy pattern for the present child node includesplanar information, in which preferably the planar information includesisPlanar information. Therein, the isPlanar information may include anisPlanar-flag indicating that the current child node likely belongs to asurface or plane defined by points of the point cloud. Therein, theisPlanar-flag is binary and indicates the presence of a plane or surfacegoing through the present child node. Further, the planar informationincludes planePosition information. Therein, the planePositioninformation may include a planePosition-flag indicating the position ofthe plane within the present child node, in particular having the values“high” and “low”, thus being also binary. Further, the planar contextinformation for the entropy encoding/decoding of the planar informationincludes context information indicating the likelihood that the presentchild node belongs to a surface defined by points of the point cloud andthe likely position of the plane within the present child node.

Preferably, the planar context information is determined according to anoccupied status of a first group of four child nodes of the neighboringparent node that are directly next to the present parent node and theoccupied status of a second group of four child nodes of the presentparent node directly opposite the first group, in which the occupiedstatus is fulfilled if at least one of the child nodes of the firstgroup or second group, respectively, is not empty. In other words, thefour child nodes of the first group of the neighboring parent node aredirectly adjacent to the common surface of the present parent node andthe neighboring parent node and the second group is within the presentparent node arranged directly adjacent to the opposite side of thepresent parent node. Each of the groups preferably define a plane by thefour child nodes, in which each of the planes is perpendicular to theconsidered geometrical axis and directly adjacent to the present childnode to be evaluated. Thus, sufficient information can be acquired toprovide a reliable prediction about the isPlanar and planePosition to beencoded.

Preferably, if the first group is occupied and the second group is notoccupied, the planar information are entropy encoded with the planarcontext information that includes the probability of a “low” plane inthe present child node. Indeed, it is likely that the present child nodedefines a surface represented by the point cloud.

Preferably, if the first group is not occupied and the second group isoccupied, the planar information are entropy encoded with the planarcontext information that includes the probability of a “high” plane inthe present child node. Indeed, it is likely that the present child nodedefines a surface represented by the point cloud.

Preferably, if the first group is occupied and the second group isoccupied, the planar information are entropy encoded with the planarcontext information that includes the probability of no plane in thepresent child node. Indeed, it is not likely that the present child nodedefines a surface represented by the point cloud.

Preferably, if the first group is not occupied and the second group isnot occupied, the planar information are entropy encoded with the planarcontext information that includes the probability of no plane in thepresent child node. Indeed, it is not likely that the present child nodedefines a surface represented by the point cloud.

In an aspect of the present invention a method for decoding a bitstreamof compressed point cloud data is provided to generate a reconstructedpoint cloud, in which the point cloud's geometry is represented by anoctree-based structure with a plurality of nodes having parent-childrelationships by recursively splitting a volumetric space containing thepoint cloud into sub-volumes each associated with a node of theoctree-bases structure, the method includes the steps of:

entropy decoding the bitstream based on the planar context informationof the present child node, in which the planar context information ofthe present child node is determined according to the occupancy patternof the present parent node and the occupancy pattern of at least oneneighboring parent node positioned directly next to the present parentnode.

Therein, the present parent node and the neighboring parent node are onthe same depth D of the tree. Due to their position, the present parentnode and the neighboring parent node share a common surface. For thecurrent child node, being a direct child of the present parent node atdepth D+1 of the tree, the planar context information is determined independence on the occupancy pattern of the present parent node and theat least one neighboring parent node. Therein, the planar contextinformation includes information about a possible plane or surfacerepresented by the point cloud going through the present child node.Thus, information about the occupancy of at least one direct neighbor ofthe present parent node is considered as context information for theentropy decoding. As a consequence, more reliable predictions about theoccupancy of the present child node can be made which can be used by theentropy decoder. In this manner, the complete tree is traversed todetermine an occupancy pattern for each node and determines the completetree-structure in order to determine the position of the points of thepoint cloud.

Preferably, the method of decoding is further built according to thefeatures described above with respect to the method for encoding. Thesefeatures can be freely combined with the method of decoding.

Other aspects and features of the present application will be understoodby those of ordinary skill in the art from a review of the followingdescription of examples in conjunction with the accompanying figures.

At times in the description below, the terms “node” and “sub-volume” maybe used interchangeably. It will be appreciated that a node isassociated with a sub-volume. The node is a particular point on the treethat may be an internal node or a leaf node. The sub-volume is thebounded physical space that the node represents. The term “volume” maybe used to refer to the largest bounded space defined for containing thepoint cloud. The volume is recursively divided into sub-volumes for thepurpose of building out a tree-structure of interconnected nodes forcoding the point cloud data.

A point cloud is a set of points in a three-dimensional coordinatesystem. The points are often intended to represent the external surfaceof one or more objects. Each point has a location (position) in thethree-dimensional coordinate system. The position may be represented bythree coordinates (X, Y, Z), which can be Cartesian or any othercoordinate system. The points may have other associated attributes, suchas color, which may also be a three-component value in some cases, suchas R, G, B or Y, Cb, Cr. Other associated attributes may includetransparency, reflectance, a normal vector, etc., depending on thedesired application for the point cloud data.

Point clouds can be static or dynamic. For example, a detailed scan ormapping of an object or topography may be static point cloud data. TheLiDAR-based scanning of an environment for machine-vision purposes maybe dynamic in that the point cloud (at least potentially) changes overtime, e.g. with each successive scan of a volume. The dynamic pointcloud is therefore a time-ordered sequence of point clouds.

Point cloud data may be used in a number of applications, includingconservation (scanning of historical or cultural objects), mapping,machine vision (such as autonomous or semi-autonomous cars), and virtualreality systems, to give some examples. Dynamic point cloud data forapplications like machine vision can be quite different from staticpoint cloud data like that for conservation purposes. Automotive vision,for example, typically involves relatively small resolution,non-coloured and highly dynamic point clouds obtained through LiDAR (orsimilar) sensors with a high frequency of capture. The objective of suchpoint clouds is not for human consumption or viewing but rather formachine object detection/classification in a decision process. As anexample, typical LiDAR frames contain on the order of tens of thousandsof points, whereas high quality virtual reality applications requireseveral millions of points. It may be expected that there will be ademand for higher resolution data over time as computational speedincreases and new applications are found.

While point cloud data is useful, a lack of effective and efficientcompression, i.e. encoding and decoding processes, may hamper adoptionand deployment.

One of the more common mechanisms for coding point cloud data is throughusing tree-based structures. In a tree-based structure, the boundingthree-dimensional volume for the point cloud is recursively divided intosub-volumes. Nodes of the tree correspond to sub-volumes. The decisionof whether or not to further divide a sub-volume may be based on theresolution of the tree and/or whether there are any points contained inthe sub-volume. A leaf node may have an occupancy flag that indicateswhether its associated sub-volume contains a point or not. Splittingflags may signal whether a node has child nodes (i.e. whether a currentvolume has been further split into sub-volumes). These flags may beentropy coded in some cases and in some cases predictive coding may beused.

A commonly-used tree structure is an octree. In this structure, thevolumes/sub-volumes are all cubes and each split of a sub-volume resultsin eight further sub-volumes/sub-cubes. An example for such atree-structure is shown in FIG. 3 having a node 30 that might representthe volume containing the complete point cloud. This volume is splitinto eight sub-volumes 32, each associated with a node in the octree ofFIG. 3 . Points in the nodes indicate occupied nodes 34 containing atleast one point 35 of the point cloud, while empty nodes 36 arerepresenting sub-volumes with no points of the point clouds. As depictedin FIG. 3 , occupied nodes might by further split into eight sub-volumesassociated with child nodes 38 of a particular parent node 40 in orderto determine the occupancy pattern of the parent node 40. As shown inFIG. 3 , the occupancy pattern of the exemplified parent node 40 mightbe represented as “00100000” in a binary form, indicating an occupiedthird child node 38. In some realizations this occupancy pattern isencoded by a binary entropy encoder to generate a bitstream of the pointcloud data.

Reference is now made to FIG. 1 , which shows a simplified block diagramof a point cloud encoder 10 in accordance with aspects of the presentapplication. The point cloud encoder 10 receives the point cloud dataand might include a tree building module for producing an octreerepresenting the geometry of the volumetric space containing point cloudand indicating the location or position of points from the point cloudin that geometry.

The basic process for creating an octree to code a point cloud mayinclude:

-   -   1. Start with a bounding volume (cube) containing the point        cloud in a coordinate system;    -   2. Split the volume into 8 sub-volumes (eight sub-cubes);    -   3. For each sub-volume, mark the sub-volume with 0 if the        sub-volume is empty, or with 1 if there is at least one point in        it;    -   4. For all sub-volumes marked with 1, repeat (2) to split those        sub-volumes, until a maximum depth of splitting is reached; and    -   5. For all leaf sub-volumes (sub-cubes) of maximum depth, mark        the leaf cube with 1 if it is non-empty, 0 otherwise.

The tree may be traversed in a pre-defined order (breadth-first ordepth-first, and in accordance with a scan pattern/order within eachdivided sub-volume) to produce a sequence of bits representing theoccupancy pattern of each node.

This sequence of bits may then be encoded using an entropy encoder 16 toproduce a compressed bitstream 14. The entropy encoder 16 may encode thesequence of bits using a context model 18 that specifies probabilitiesfor coding bits based on a context determination by the entropy encoder16. The context model 18 may be adaptively updated after coding of eachbit or defined set of bits.

Like with video or image coding, point cloud coding can includepredictive operations in which efforts are made to predict the patternfor a sub-volume, and the residual from the prediction is coded insteadof the pattern itself. Predictions may be spatial (dependent onpreviously coded sub-volumes in the same point cloud) or temporal(dependent on previously coded point clouds in a time-ordered sequenceof point clouds).

A block diagram of an example point cloud decoder 20 that corresponds tothe encoder 10 is shown in FIG. 2 . The point cloud decoder 20 includesan entropy decoder 22 using the same context model 24 used by theencoder 10. The entropy decoder 22 receives the input bitstream 26 ofcompressed data and entropy decodes the data to produce an outputsequence of decompressed bits. The sequence is then converted intoreconstructed point cloud data by a tree reconstructor. The treereconstructor rebuilds the tree structure 28 from the decompressed dataand knowledge of the scanning order in which the tree data wasbinarized. The tree reconstructor is thus able to reconstruct thelocation of the points from the point cloud.

Therein, the occupancy pattern might include planar information aboutthe probability that a certain node is occupied since the point in thisnode belongs to a surface. Usually, the real world is dominated byclosed surfaces. This is in particular true for indoor rooms but alsofor urban outdoor scenes. This fact is used by the entropy encoder anddecoder. If a surface represented by the point cloud can be detected,predictions about the distribution of point on this surface can be madeand thus a probability for the occupancy of a certain node belonging tothis surface can be made. This might be done by defining anisPlanar-flag that will be encoded in the bitstream. The isPlanar-flagis usually a binary value wherein a set isPlanar-flag (isPlanar =1) fora certain node is interpreted that there is a prevailing likelihood thatthis node belongs to a certain surface. In addition to the mere fact ofthe presence of a surface in a node, further planar information might beconsidered such as planePosition-flag indicating the position of theplane within the present child node. planePosition-flag might also be abinary value, having the values “high” and “low” referring to therespective position. This planar information is encoded to the bitstreamby usage of the planar context information by the entropyencoder/decoder thereby reducing the data of the bitstream.

Referring to FIG. 4 , showing a parent node 112 split into its eightchild nodes 110 being 2×2×2 cubes with each having the same size andwith an edge length being half the edge length of the cube associatedwith the parent node 112. Further, FIG. 4 indicates the used numberingof the child nodes 110 within a parent node 112. The numbering systemshown in FIG. 4 will be used in the further explanation. Therein, FIG. 4also indicates the spatial orientation of the shown parent node 112 inthe three-dimensional space indicated by the geometrical axis X, Y, Z.

Now referring to FIG. 5 showing a present parent node 100 with two childnodes 102 exemplified as occupied nodes, i.e. the sub-volumes of the twochild nodes contain each at least one point of the point cloud. Further,indicated in FIG. 5 are neighboring parent nodes 104, 106, 108. Therein,prior to predicting the planar context information of the child nodes102 at a certain octree-level, occupation information of the threeadjacent neighboring parent nodes 104, 106, 108 at the same level D asthe present parent node 100 is known at the decoder side. Due to thegiven decoding order, upon evaluating the child node 102 of the presentparent node 100 information about the neighboring parent nodes 104, 108,106 are already known at the decoder side and can be used to determinethe planar context information. Since these three neighboring parentnodes 104, 106, 108 are adjacent to the present child node 102, thereexists a correlation between the occupancy of these neighboring parentnodes 104, 106, 108 and the present child node 102. This information iseasy to obtain but currently not used in existing encoding and decodingmethods. For example, if evaluating the planar context information ofthe present child node 102 along the X-axis, the directly adjacentneighboring parent node 104 along the X-axis might be considered.Further, as explained above, the directly adjacent parent node isconsidered which is in the order of decoding prior to the present parentnode. Thus, during decoding the occupancy of the prior nodes are knownby the decoder and can be used to determine planar context information.If the planar context information is evaluated along the Y-axis, theneighboring parent node 106 along the Y-axis next to the present parentnode 100 is considered in order of decoding. This information is alsoknown to the decoder and can be easily used to acquire reliable planarcontext information for the present child node 102.

Now referring to FIG. 6 showing the configuration of how to determineplanar context information which then can be used in the entropyencoding in order to compress the point cloud data. In this regard, itis an object of the present invention to consider the occupancy of theparent neighbors to the present child node 120 and to incorporate suchinformation in the planar context information when coding the planarinformation, i.e. the isPlanar-flag or planePosition-flag.

FIG. 6 (a) shows the determination of the planar context informationalong the X-axis. The planar context information of the present childnode 120 shall be evaluated. Therefore, the occupancy of the presentparent node 122 as well as the neighboring parent node 123 directlyadjacent to the present parent node 122 is evaluated. Therein, theneighboring parent node 124 is arranged along the X-axis which is to beevaluated. Further, the neighboring parent node 124 is arranged prior tothe present parent node 122 in order of decoding. Thus, upon decodingthe present parent node 122, all information about the neighboringparent node 124 is already available. In addition, also during codingthe information of the present child node 120 occupancy informationabout the neighboring parent node 124 is also known, since positions andthus occupancy pattern of the point cloud is completely known to theencoder.

In particular, in order to determine the planar context information ofthe present child node 120 a first group 126 of child nodes, being childnodes of the neighboring parent node 124, and a second group 128, beingchild nodes of the present parent node 122 are considered. Therein, thefour child nodes of the first group 126 of the neighboring parent node124 are directly next and adjacent to the present parent node. Thus, thechild nodes of the first group 126 each share a common surface with thepresent parent node 122. The second group 128 of the four child nodes ofthe present parent node 122 is directly opposite the first group. Thus,by the four child nodes of the second group 128 a side face of thepresent parent node 122 is formed which is directly opposite to the sideface which is the common surface between the neighboring parent node 124and the present parent node 122. In other words, by the first group 126and the second group 128 planes are defined which are on both sides ofthe present child node 120 and which are perpendicular to the axis underconsideration. Thus, by these two planes a reliable determination of theplanar context information can be provided. A surface in the point cloudgoing through the present child node 120 is also highly likely goingthrough one of the planes defined by the first group 126 or the secondgroup 128 and thus the planar context information of the present childnode 120 strongly depends on the occupied status of the first group 126and/or the occupied status of the second group 128. Therein, the firstgroup 126 and the second group 128 are considered “occupied” if at leastone child node in the first group and/or second group are not empty andcontain a point of the point cloud.

Generally speaking, if the first group is occupied and the second groupis not occupied the isPlanar-flag and the planePosition-flag are entropycoded as planar information with a likelihood of a “low” plane in thepresent child node as planar context information. If the first group isnot occupied and the second group is occupied, the isPlanar-flag and theplanePosition-flag are entropy coded as planar information with alikelihood of a “high” plane in the present child as planar contextinformation. If the first group is occupied and the second group isoccupied, the isPlanar-flag and the planePosition-flag are entropy codedas planar information with a likelihood of no plane in the present childnode as planar context information. In this situation it can be assumedthat a surface goes through the first group 126 and a second surfacegoes through the second group 128 rendering it unlikely that in betweena further and additional surface is present. Therein, it should be notedthat this evaluation is only along one axis. A plane or surface goingalong the X-axis in the X-Z-plane for example would not be consideredduring evaluation of the X-axis but during evaluation along the Y-axisfor example. If the first group is not occupied and the second group isnot occupied, then the isPlanar-flag and the planePosition flag areentropy coded as planar information with a likelihood of no plane in thepresent child node as planar context information, since it is highlyunlikely that an isolated surface is only within the present child node120.

More specifically speaking, for the evaluation along the X-axis togetherwith the numbering system shown in FIG. 4 :

-   -   If sub-nodes 4 to 7 of the first group 126 of the neighboring        parent node 124 are occupied and sub-nodes 4 to 7 of the present        parent node 122 are not occupied, it is strongly likely that        there is a plane in the present child node 120 and the plane        position is low. Thus, for the coding of the isPlanar and        planePosition information, the probability of a “low” plane        perpendicular to the X-axis in the present child node is        considered as planar context information when performing entropy        coding.    -   If sub-nodes 4 to 7 of the first group 126 of the neighboring        parent node 124 are not occupied (i.e., empty) and sub-nodes 4        to 7 of the present parent node 122 are occupied, it is strongly        likely that there is a plane in the present child node 120 and        the plane position is high. Thus, for the coding of the isPlanar        and planePosition information, the probability of a “high” plane        perpendicular to the X-axis in the present child node is        considered as planar context information when performing entropy        coding.    -   If sub-nodes 4 to 7 of the first group 126 of the neighboring        parent node 124 are occupied and sub-nodes 4 to 7 of the present        parent node 122 are occupied, it is strongly likely that current        node is not eligible to planar mode. Thus, for the coding of the        isPlanar and planePosition information, the probability of an        absence of a plane perpendicular to the X-axis in the present        child node is considered as planar context information when        performing entropy coding.    -   If sub-nodes 4 to 7 of the first group 126 of the neighboring        parent node 124 are not occupied and sub-nodes 4 to 7 of the        present parent node 122 are not occupied neither, it is strongly        unlikely that current node can be eligible to planar mode. Thus,        for the coding of the isPlanar and planePosition information,        the probability of an absence of a plane perpendicular to the        X-axis in the present child node is considered as planar context        information when performing entropy coding.

For evaluating along Y-axis in accordance to FIG. 6 (b) in order todetermine the planar context information of the present child node 120it is set the following:

-   -   If sub-nodes 2,3,6,7 of the first group 126 of the neighboring        parent node 124 are occupied and sub-nodes 2,3,6,7 of the        present parent node 122 are not occupied, it is strongly likely        that there is a plane in the present child node 120 and the        plane position is low. Thus, for the coding of the isPlanar and        planePosition information, the probability of a “low” plane        perpendicular to the Y-axis in the present child node is        considered as planar context information when performing entropy        coding.    -   If sub-nodes 2,3,6,7 of the first group 126 of the neighboring        parent node 124 are not occupied (i.e., empty) and sub-nodes        2,3,6,7 of the present parent node 122 are occupied, it is        strongly likely that there is a plane in the present child node        120 and the plane position is high. Thus, for the coding of the        isPlanar and planePosition information, the probability of a        “high” plane perpendicular to the Y-axis in the present child        node is considered as planar context information when performing        entropy coding.    -   If sub-nodes 2,3,6,7 of the first group 126 of the neighboring        parent node 124 are occupied and sub-nodes 2,3,6,7 of the        present parent node 122 are occupied, it is strongly likely that        current node is not eligible to planar mode. Thus, for the        coding of the isPlanar and planePosition information, the        probability of an absence of a plane perpendicular to the Y-axis        in the present child node is considered as planar context        information when performing entropy coding.    -   If sub-nodes 2,3,6,7 of the first group 126 of the neighboring        parent node 124 are not occupied and sub-nodes 2,3,6,7 of the        present parent node 122 are not occupied neither, it is strongly        unlikely that current node can be eligible to planar mode. Thus,        for the coding of the isPlanar and planePosition information,        the probability of an absence of a plane perpendicular to the        Y-axis in the present child node is considered as planar context        information when performing entropy coding.

For evaluating along the Z-axis in accordance to FIG. 6 (c) in order todetermine the planar context information of the present child node 120it is set the following:

-   -   If sub-nodes 1,3,5,7 of the first group 126 of the neighboring        parent node 124 are occupied and sub-nodes 1,3,5,7 of the        present parent node 122 are not occupied, it is strongly likely        that there is a plane in the present child node 120 and the        plane position is low. Thus, for the coding of the isPlanar and        planePosition information, the probability of a “low” plane        perpendicular to the Z-axis in the present child node is        considered as planar context information when performing entropy        coding.    -   If sub-nodes 1,3,5,7 of the first group 126 of the neighboring        parent node 124 are not occupied (i.e., empty) and sub-nodes        1,3,5,7 of the present parent node 122 are occupied, it is        strongly likely that there is a plane in the present child node        120 and the plane position is high. Thus, for the coding of the        isPlanar and planePosition information, the probability of a        “high” plane perpendicular to the Z-axis in the present child        node is considered as planar context information when performing        entropy coding.    -   If sub-nodes 1,3,5,7 of the first group 126 of the neighboring        parent node 124 are occupied and sub-nodes 1,3,5,7 of the        present parent node 122 are occupied, it is strongly likely that        current node is not eligible to planar mode. Thus, for the        coding of the isPlanar and planePosition information, the        probability of an absence of a plane perpendicular to the Y-axis        in the present child node is considered as planar context        information when performing entropy coding.    -   If sub-nodes 1,3,5,7 of the first group 126 of the neighboring        parent node 124 are not occupied and sub-nodes 1,3,5,7 of the        present parent node 122 are not occupied neither, it is strongly        unlikely that current node can be eligible to planar mode. Thus,        for the coding of the isPlanar and planePosition information,        the probability of an absence of a plane perpendicular to the        Y-axis in the present child node is considered as planar context        information when performing entropy coding.

As a summary, in all cases, a reliable planar context information modelis established which can be used for the entropic encoding in order toconsider the occurrence of a surface or plane in the respective nodesand use this knowledge to validly predict occupancy of nodes for datacompression by the entropy encoder/decoder.

In addition to the above described consideration with respect to FIG. 6, the isPlanar-flag might be coded depending on further requirements.These further requirements may include for example: the present parentnode 122 of the present child node 120 is considered isPlanar, and theoccupancy pattern of the present parent node's neighbor adjacent to thecurrent node in a direction perpendicular to the axis underconsideration, i.e. the parent vertical neighbor adjacent to the presentparent node 122 for the X-axis, is not empty and a distance d from theclosest already coded node at the same coordinate of the axis currentlyunder evaluation at the same depth D within the tree structure isconsidered as isPlanar. Further requirements may also be added in orderto increase the reliability of the planar context information used bythe entropic encoder and decoder.

Referring now to FIG. 7 (a) showing a method for encoding a point cloudincluding the steps of determining an occupancy pattern for a parentnode based on the occupancy of its child nodes (step 110);

Determining a planar context information for at least one of the childnodes, wherein the planar context information of a present child node isdetermined according to the occupancy pattern of the present parent nodeand the occupancy pattern of at least one neighboring parent nodepositioned directly next to the present parent node (step 120); and

entropy encoding the occupancy patterned based on the determined planarcontext information to produce encoded data for its tree (step 130).

Referring now to FIG. 7 (b) showing a method for decoding a point cloudincluding the steps of entropy decoding the bitstream based on theplanar context information of the present child node, wherein the planarcontext information of the present child node is determined according tothe occupancy pattern of the present parent node and the occupancypattern of at least one neighboring parent node positioned directly nextto the present parent node (Step 210).

Thus, in accordance to the present invention, information about theneighboring parent node are used and evaluated in order to acquire aplanar context information model more reliably predicting the occurrenceof a surface within the present child node. This information can be usedin order to reduce the amount of data of the compressed point clouddata, wherein the same planar context information model is also used forthe decoder in order to be able to reliably decode the provide bitstreamwith high efficiency and accuracy.

Thereby, significant data reduction of at least 1% can be achieved withrespect to prior encoding methods. However, this value is dependent onthe density of the points wherein in particular for indoor point clouds,such as LiDAR point clouds it is assumed that they can be considered asa dense point cloud. FIG. 8 shows different examples for model sequenceswherein the encoded according to the present invention is compared tothe TMC13 v.9 encoder (also called G-PCC) of the MPEG-3DG group. Asshown data reduction of around 1% can be achieved by the presentinvention. The quality of the reconstructed point cloud with and withoutthe proposed technique is exactly the same.

In embodiments of the encoding or decoding method, the neighboringparent node is the node directly next to the current child node, sharingone surface with the current child node.

In embodiments of the encoding or decoding method, the neighboringparent node is arranged along one of the geometrical axes.

In embodiments of the encoding or decoding method, the planar contextinformation is determined along all three geometrical axes.

In embodiments of the encoding or decoding method, the present parentnode is arranged relatively to the neighboring parent node in order ofdecoding.

In embodiments of the encoding or decoding method, the occupancy patternfor the present child node includes planar information, whereinpreferably the planar information includes isPlanar information andplanePosition information, and the planar context information for theentropy encoding/decoding of the planar information includes contextinformation indicating the likelihood that the present child nodebelongs to a surface defined by points of the point cloud and the likelyposition of the plane within the present child node.

In embodiments of the encoding or decoding method, the planar contextinformation is determined according to an occupied status of a firstgroup of four child nodes of the neighboring parent node that aredirectly next to the present parent node and the occupied status of asecond group of four child nodes of the present parent node directlyopposite the first group wherein the occupied status is fulfilled if atleast one of the child nodes of the first group or second group,respectively, is not empty.

In embodiments of the encoding or decoding method, if the first group isoccupied and the second group is not occupied, the planar information isentropy encoded/decoded with the planar context information thatincludes the likelihood of a “low” plane.

In embodiments of the encoding or decoding method, if the first group isnot occupied and the second group is occupied, the planar information isentropy coded/decoded with the planar context information that includesthe likelihood of a “high” plane.

In embodiments of the encoding or decoding method, if the first group isoccupied and the second group is occupied, the planar information isentropy coded/decoded with the planar context information that includesthe likelihood of no plane.

In embodiments of the encoding or decoding method, if the first group isnot occupied and the second group is not occupied, the planarinformation is entropy coded/decoded with the planar context informationthat includes the likelihood of no plane.

In a preferred embodiment the method for encoding/decoding a point cloudto generate a bitstream of compressed point cloud data is implemented ina LIDAR (Light detection and ranging) device. The LIDAR device includesa light transmitting module and a sensor module. Therein, the lighttransmitting module is configured to scan the environment with laserlight and an echo of the laser light reflected by objects in theenvironment is measured with a sensor of the sensor module. Further, theLIDAR device includes an evaluation module configured to determine a 3Drepresentation of the environment in a point cloud preferably bydifferences in laser return times and/or wavelengths of the reflectedlaser light. Thereby, the echo may include up to millions of points ofposition information of the objects or environment resulting in largepoint clouds increasing the demands on computational devices to furtherprocess or evaluating this point clouds. In certain applications such asautonomous driving, processing of the LIDAR point cloud must be almostin real-time due to safety requirements. Thus, efficient and accuratecompression of the point could data is necessary. Therefore, the LIDARdevice may includes an encoder including a processor and a memorystorage device. The memory storage device may store a computer programor application containing instructions that, when executed, cause theprocessor to perform operations such as those described herein. Forexample, the instructions may encode and output bitstreams encoded inaccordance with the methods described herein. Additionally oralternatively the LIDAR device may includes a decoder including aprocessor and a memory storage device. The memory storage device mayinclude a computer program or application containing instructions that,when executed, cause the processor to perform operations such as thosedescribed herein. Thus, by the encoder/decoder efficient and accuratecompression of the point cloud data is enabled, providing thepossibility to handle the acquired point cloud data more efficiently andpreferably in real-time. Preferably, the processor of the encoder andthe processor of the decoder are the same. Preferably, the memorystorage device of the encoder and the memory storage device of thedecoder are the same. Preferably the processor of the encoder and/ordecoder are further configured to further process or evaluate the pointcloud even more preferably in real-time. In particular, for the exampleof autonomous driving, evaluation of the point cloud could includedetermination of obstacles in the direction of driving.

Reference is now made to FIG. 8 , which shows a simplified block diagramof an example embodiment of an encoder 1100. The encoder 1100 includes aprocessor 1102 and a memory storage device 1104. The memory storagedevice 1104 may store a computer program or application containinginstructions that, when executed, cause the processor 1102 to performoperations such as those described herein. For example, the instructionsmay encode and output bitstreams encoded in accordance with the methodsdescribed herein. It will be understood that the instructions may bestored on a non-transitory computer-readable medium, such as a compactdisc, flash memory device, random access memory, hard drive, etc. Whenthe instructions are executed, the processor 1102 carries out theoperations and functions specified in the instructions so as to operateas a special-purpose processor that implements the describedprocess(es). Such a processor may be referred to as a “processorcircuit” or “processor circuitry” in some examples.

Reference is now also made to FIG. 9 , which shows a simplified blockdiagram of an example embodiment of a decoder 1200. The decoder 1200includes a processor 1202 and a memory storage device 1204. The memorystorage device 1204 may include a computer program or applicationcontaining instructions that, when executed, cause the processor 1202 toperform operations such as those described herein. It will be understoodthat the instructions may be stored on a computer-readable medium, suchas a compact disc, flash memory device, random access memory, harddrive, etc. When the instructions are executed, the processor 1202carries out the operations and functions specified in the instructionsso as to operate as a special-purpose processor that implements thedescribed process(es) and methods. Such a processor may be referred toas a “processor circuit” or “processor circuitry” in some examples.

It will be appreciated that the decoder and/or encoder according to thepresent application may be implemented in a number of computing devices,including, without limitation, servers, suitably programmed generalpurpose computers, machine vision systems, and mobile devices. Thedecoder or encoder may be implemented by way of software containinginstructions for configuring a processor or processors to carry out thefunctions described herein. The software instructions may be stored onany suitable non-transitory computer-readable memory, including CDs,RAM, ROM, Flash memory, etc.

It will be understood that the decoder and/or encoder described hereinand the module, routine, process, thread, or other software componentimplementing the described method/process for configuring the encoder ordecoder may be realized using standard computer programming techniquesand languages. The present application is not limited to particularprocessors, computer languages, computer programming conventions, datastructures, other such implementation details. Those skilled in the artwill recognize that the described processes may be implemented as a partof computer-executable code stored in volatile or non-volatile memory,as part of an application-specific integrated chip (ASIC), etc.

The present application also provides for a computer-readable signalencoding the data produced through application of an encoding process inaccordance with the present application.

Certain adaptations and modifications of the described embodiments canbe made. Therefore, the above discussed embodiments are considered to beillustrative and not restrictive. In particular, embodiments can befreely combined with each other.

1. A method for encoding a point cloud to generate a bitstream ofcompressed point cloud data, wherein the point cloud's geometry isrepresented by an octree-based structure with a plurality of nodeshaving parent-child relationships by recursively splitting a volumetricspace containing the point cloud into sub-volumes each associated with anode of the octree-based structure, comprising the steps: determining anoccupancy pattern for a parent node based on the occupancy of its childnodes; determining a-planar context information for at least one of thechild nodes, wherein the planar context information of a present childnode is determined according to the occupancy pattern of the presentparent node and the occupancy pattern of at least one neighboring parentnode positioned directly next to the present parent node; and entropyencoding the occupancy pattern of the parent node based on thedetermined planar context information to produce encoded data for thebitstream.
 2. A method for decoding a bitstream of compressed pointcloud data to generate a reconstructed point cloud, wherein the pointcloud's geometry is represented by an octree-based structure with aplurality of nodes having parent-child relationships by recursivelysplitting a volumetric space containing the point cloud into sub-volumeseach associated with a node of the octree-bases structure, comprisingthe steps: entropy decoding the bitstream based on the planar contextinformation of a present child node, wherein the planar contextinformation of the present child node is determined according to anoccupancy pattern of the a present parent node and an occupancy patternof at least one neighboring parent node positioned directly next to thepresent parent node.
 3. The method according to claim 1, characterizedin that the neighboring parent node is a node directly next to presentcurrent child node, sharing one surface with the present child node. 4.The method according to claim 1, characterized in that the neighboringparent node is arranged along one of geometrical axes.
 5. The methodaccording to claim 1, characterized in that the planar contextinformation is determined along all three geometrical axes.
 6. Themethod according to claim 1, characterized in that the present parentnode is arranged relatively to the neighboring parent node in order ofdecoding
 7. The method according to claim 1, characterized in that theoccupancy pattern for the present child node comprises planarinformation, wherein the planar information comprises isPlanarinformation and planePosition information, and the planar contextinformation for the entropy encoding of the planar information includescontext information indicating a likelihood that the present child nodebelongs to a surface defined by points of the point cloud and a likelyposition of the plane within the present child node.
 8. The methodaccording to claim 7, characterized in that the planar contextinformation is determined according to an occupied status of a firstgroup of four child nodes of the neighboring parent node that aredirectly next to the present parent node and the an occupied status of asecond group of four child nodes of the present parent node directlyopposite the first group, wherein the occupied status is fulfilled if atleast one of the child nodes of the first group or second group,respectively, is not empty.
 9. The method according to claim 8,characterized in that if the first group is occupied and the secondgroup is not occupied, the planar information is entropy encoded withthe planar context information that includes the likelihood of a “low”plane; or if the first group is not occupied and the second group isoccupied, the planar information is entropy coded with the planarcontext information that includes the likelihood of a “high” plane; orif the first group is occupied and the second group is occupied, theplanar information is entropy coded with the planar context informationthat includes the likelihood of no plane; or if the first group is notoccupied and the second group is not occupied, the planar information isentropy coded/decoded with the planar context information that includesthe likelihood of no plane.
 10. An encoder for encoding a point cloud togenerate a bitstream of compressed point cloud data, wherein the pointcloud's geometry is represented by an octree-based structure with aplurality of nodes having parent-child relationships by recursivelysplitting a volumetric space containing the point cloud into sub-volumeseach associated with a node of the octree-based structure, the encodercomprising: a processor and a memory storage device, wherein in thememory storage device instructions executable by the processor arestored that, when executed, cause the processor to perform thefollowing: determining an occupancy pattern for a parent node based onthe occupancy of its child nodes; determining planar context informationfor at least one of the child nodes, wherein the planar contextinformation of a present child node is determined according to theoccupancy pattern of the present parent node and the occupancy patternof at least one neighboring parent node positioned directly next to thepresent parent node; and entropy encoding the occupancy pattern of theparent node based on the determined planar context information toproduce encoded data for the bitstream.
 11. A decoder for decoding abitstream of compressed point cloud data to generate a reconstructedpoint cloud, wherein the point cloud's geometry is represented by anoctree-based structure with a plurality of nodes having parent-childrelationships by recursively splitting a volumetric space containing thepoint cloud into sub-volumes each associated with a node of theoctree-bases structure, the decoder comprising: a processor and a memorystorage device, wherein in the memory storage device instructionsexecutable by the processor are stored that, when executed, cause theprocessor to perform the method according to claim
 2. 12. (canceled).13. The method according to claim 2, characterized in that theneighboring parent node is the node directly next to the current childnode, sharing one surface with the current child node.
 14. The methodaccording to claim 2, characterized in that the neighboring parent nodeis arranged along one of geometrical axes.
 15. The method according toclaim 2, characterized in that the planar context information isdetermined along all three geometrical axes.
 16. The method according toclaim 2, characterized in that the present parent node is arrangedrelatively to the neighboring parent node in order of decoding
 17. Themethod according to claim 2, characterized in that the occupancy patternfor the present child node comprises planar information, wherein theplanar information comprises isPlanar information and planePositioninformation, and the planar context information for the entropy decodingof the planar information includes context information indicating alikelihood that the present child node belongs to a surface defined bypoints of the point cloud and a likely position of the plane within thepresent child node.
 18. The method according to claim 17, characterizedin that the planar context information is determined according to anoccupied status of a first group of four child nodes of the neighboringparent node that are directly next to the present parent node and anoccupied status of a second group of four child nodes of the presentparent node directly opposite the first group, wherein the occupiedstatus is fulfilled if at least one of the child nodes of the firstgroup or second group, respectively, is not empty.
 19. The methodaccording to claim 18, characterized in that if the first group isoccupied and the second group is not occupied, the planar information isentropy decoded with the planar context information that includes thelikelihood of a “low” plane; or if the first group is not occupied andthe second group is occupied, the planar information is entropy decodedwith the planar context information that includes the likelihood of a“high” plane; or if the first group is occupied and the second group isoccupied, the planar information is entropy decoded with the planarcontext information that includes the likelihood of no plane; or if thefirst group is not occupied and the second group is not occupied, theplanar information is entropy decoded with the planar contextinformation that includes the likelihood of no plane.